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Abstract. This is a project on Artificial Life where we simulate an 
ecosystem that allows cooperative interaction between agents, including 
intra-specific predator-warning communication in a virtual environment 
of predatory events. We propose, based on Peircean semiotics and in- 
formed by neuroethological constraints, an experiment to simulate the 
emergence of symbolic communication among artificial creatures. Here 
we describe the simulation environment and the creatures’ control archi- 
tectures, and briefly present obtained results. 
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1 Introduction 


According to the semiotics of C.S.Peirce, there are three fundamental kinds of 
signs underlying meaning processes: icons, indexes and symbols (CP 2.275!). 
Icons are signs that stand for their objects through similarity or resemblance 
(CP 2.276, 2.247, 8.335, 5.73); indexes are signs that have a spatio-temporal 
physical correlation with its object (CP 2.248, see 2.304); symbols are signs 
connected to O by the mediation of I. For Peirce (CP 2.307), a symbol is “A 
Sign which is constituted a sign merely or mainly by the fact that it is used and 
understood as such, whether the habit is natural or conventional, and without 
regard to the motives which originally governed its selection.” 

Based on this framework, Queiroz and Ribeiro [2] performed a neurosemiotic 
analysis of vervet monkeys’ intra-specific communication. These primates use 
vocal signs for intra-specific social interactions, as well as for general alarm 
purposes regarding imminent predation on the group [3]. They vocalize basically 
three predator-specific alarm calls which produce specific escape responses: alarm 
calls for terrestrial predators (such as leopards) are followed by a escape to the 
top of trees, alarm calls for aerial raptors (such as eagles) cause vervets to hide 
under bushes, and alarm calls for ground predators (such as snakes) elicit careful 
scrutiny of the surrounding terrain. Queiroz and Ribeiro[2] identified the different 
signs and the possible neuroanatomical substrates involved. Icons correspond to 
neural responses to the physical properties of the visual image of the predator 


1 The work of C.S.Peirce[1] is quoted as CP, followed by the volume and paragraph. 
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and the alarm-call, and exist within two independent primary representational 
domains (visual and auditory). Indexes occur in the absence of a previously 
established relationship between call and predator, when the call simply arouses 
the receiver’s attention to any concomitant event of interest, generating a sensory 
scan response. If the alarm-call operates in a sign-specific way in the absence 
of an external referent, then it is a symbol of a specific predator class. This 
symbolic relationship implies the association of at least two representations of a 
lower order in a higher-order representation domain. 


2 Simulating Artificial Semiotic Creatures 


The framework (above) guided our experiments of simulating the emergence of 
symbolic alarm calls?. The environment is bi-dimensional having approximately 
1000 by 1300 positions. The creatures are autonomous agents, divided into preys 
and predators. There are objects such as trees (climbable objects) and bushes 
(used to hide), and three types of predators: terrestrial predator, aerial predator 
and ground predator. Predators differentiate by their visual limitations: terres- 
trial predators can’t see preys over trees, aerial predators can’t see preys under 
bushes, but ground predators don’t have these limitations. The preys can be 
teachers, which vocalizes pre-defined alarms to predators, or learners, which 
try to learn these associations. There is also the self-organizer prey, which is a 
teacher and a learner at the same time, able to create, vocalize and learn alarms, 
simultaneously. 

The sensory apparatus of the preys include hearing and vision; predators 
have only a visual sensor. The sensors have parameters that define sensory areas 
in the environment, used to determine the stimuli the creatures receive. Vision 
has a range, a direction and an aperture defining a circular section, and hearing 
has just a range defining a circular area. These parameters are fixed, with ex- 
ception to visual direction, changed by the creature, and visual range increased 
during scanning. The received stimuli correspond to a number, which identifies 
the creature or object seen associated with the direction and distance from the 
stimulus’ receiver. 

The creatures have interactive abilities, high-level motor actions: adjust vi- 
sual sensor, move, attack, climb tree, hide on bush, and vocalize. These last three 
actions are specific to preys, while attacks are only done by predators. The crea- 
tures can perform actions concomitantly, except for displacement actions (move, 
attack, climb and hide) which are mutually exclusive. The move action changes 
the creature position in the environment and takes two parameters velocity (in 
positions/interaction, limited to a maximum velocity) and a direction (0-360 de- 
grees). The visual sensor adjustment modifies the direction of the visual sensor 
(and during scanning, doubles the range), and takes one parameter, the new 
direction (0-360 degrees). The attack action has one parameter that indicates 
the creature to be attacked, that must be within action range. If successful the 


? The simulator is called Symbolic Creatures Simulation. For more technical details, 
see http://www.dca.fee.unicamp.br/projects/artcog/symbcreatures 
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attack increases an internal variable, number of attacks suffered, from the at- 
tacked creature. The climb action takes as a parameter the tree to be climbed, 
that must be within the action range. When up in a tree, an internal variable 
called ‘climbed’ is set to true; when the creature moves it is turned to false and 
it goes down the tree. Analogously, the hide action has the bush to be used to 
hide as a parameter, and it uses an internal variable called ‘hidden’. The vocalize 
action has one parameter the alarm to be emitted, a number between 0 and 99, 
and it creates a new element in the environment that lasts just one interaction, 
and is sensible by creatures having hearing sensors. 

To control their actions after receiving the sensory input, the creatures have 
a behavior-based architecture [4], dedicated to action selection [5]. Our control 
mechanism is composed of various behaviors and drives. Behaviors are indepen- 
dent and parallel modules that are activated at different moments depending on 
the sensorial input and the creature’s internal state. At each iteration, behaviors 
provide their motivation value (between 0 and 1), and the one with highest value 
is activated and provides the creature actions at that instant. Drives define basic 
needs, or ‘instincts’, such as ‘fear’ or ‘hunger’, and they are represented by nu- 
meric values between 0 and 1, updated based on the sensorial input or time flow. 
This mechanism is not learned by the creature, but rather designed, providing 
basic responses to general situations. 


Predators’ Cognitive Architecture 


The predators have a simple control architecture with basic behaviors and drives. 
The drives are hunger and tiredness, and the behaviors are wandering, rest and 
prey chasing. The drives are implemented as follows: 


hunger(0) = 1 
hingeta t= { 0.01, if it attacked a prey 


ramp1(hunger(t) + 0.01.hunger(t)), otherwise 


0,2 <0 
where rampi(z) = 4 z,0<a<1 
1,e>1 


ramp (tiredness(t) — 0.1), 
velocity(t) = 0 
- velocity(t) 
rampi (tiredness(t) + 0.05. a) ‘ 
velocity(t) > maximum velocity /2 


tiredness(t + 1) = 


where velocity(t) is the creature’s velocity at the current instant (t). 


The wandering behavior has a constant motivation value of 0.4, and makes 
the creature basically move at random direction and velocity, directing its vi- 
sion toward movement direction. The resting behavior makes the creature stop 
moving and its motivation is given by 
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tiredness(t), tiredness(t) > 0.5 
rest.motivation(t) = 4 0.5, velocity(t) = Oandtiredness(t) > 0 
0, otherwise 
The behavior chasing makes the predator move towards the prey, if its out 
of range, or attack it, otherwise. The motivation of this behavior is given by 


hunger(t), hunger(t) > 0.5 and a prey is seen 


chasing-motivation(t) = k HAE 
? 


Preys’ Cognitive Architecture 


Preys have two sets of behavior: communication related behaviors and general 
behaviors. The communication related behaviors are vocalizing, scanning, asso- 
ciative learning and following, the general ones are wandering, resting and fleeing. 
Associated with these behaviors, there are different drives: boredom, tiredness, 
solitude, fear and curiosity. The learner and the teacher don’t have the same 
architecture, only teachers have the vocalize behavior and only learners have 
the associative learning behavior, the scanning behavior and the curiosity drive 
(figure 1). On the other hand, the self-organizer prey has all behaviors and drives. 
The prey’s drives are specified by the expressions 


ramp2(boredom(t) + rate - boredom(t)), velocity(t) = 0 


boredorm(t +1) = { ramp2(boredom(t) — 0.1), otherwise 


0.05, hidden on bush or climbed on tree 


where rate = { G4. othierwisé and 
0.01, z < 0.01 
ramp2(x) = x, 0.01 <x < 0.99 
0.99, x > 0.99 


_ J rampe2(1.0), if predator was seen 
feart +1) = { ramp2(fear(t) — 0.05(1.0 — fear(t)), otherwise 


rampa2(solitude(t) + 0.1.solitude(t)), no prey is seen 


soldat) = { rampa2(solitude(t) — 0.1(1.0 — solitude(t))), otherwise 


curiosity(t) = ramp2(maz; strengthW M;(t)), where i ce W M Aud 


The tiredness drive is computed by the same expression used by predators. 

The vocalize behavior and associative learning behavior can run in parallel 
with all other behaviors, so it does not undergo behavior selection. The vocalize 
behavior makes the prey emit an alarm when a predator is seen. The teacher 
has a fixed alarm set, using alarm number 1 for terrestrial predator, 2 for aerial 
predator and 3 for ground predator. The self-organizer uses the alarm with the 
highest association value in the associative memory (next section), or chooses 
randomly an alarm from 0 to 99 and places it in the associative memory, if none 
is known. (The associative learning behavior is described in the next section.) 
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Fig. 1. Preys’ cognitive architecture: (a) learners have scanning and associative learn- 
ing capabilities and (b) teachers have vocalizing capability. The self-organizer prey is 
a teacher and a learner at the same time and has all these behaviors. 


The scanning behavior makes the prey turn towards the alarm emitter di- 
rection and move at this direction, if an alarm is heard, turn to the same vision 
direction of the emitter, but still moving towards the emitter, if the emitter is 
seen, or keep the same vision and movement direction, if the alarm is not heard 
anymore. The motivation is given by curiosity(t), if an alarm is heard or if cu- 
riosity(t)30.2. This behavior also makes the vision range double, simulating a 
wide sensory scanning process. 

To keep preys near each other and not spread out in the environment, the fol- 
lowing behavior makes the prey keep itself between a maximum and a minimum 
distance of another prey, by moving towards or away from it. This was inspired 
by experiments in simulation of flocks, schools and herds. The motivation for 
following is equal to solitude(t), if another prey is seen. 

The fleeing behavior has its motivation given by fear(t). It makes the prey 
move away from the predator with maximum velocity, or in some situations, 
perform specific actions depending upon the type of prey. If a terrestrial predator 
is or was just seen and there’s a tree not near the predator (the difference between 
predator direction and tree direction is more than 60 degrees), the prey moves 
toward the tree and climbs it. If it is an aerial predator and there’s a bush not 
near it, the prey moves toward the bush and hides under it. If the predator is not 
seen anymore, and the prey is not up on a tree or under a bush, it keeps moving 
in the same direction it was before, slightly changing its direction at random. 

The wandering behavior makes the prey move at a random direction and 
velocity, slightly changing it at random. The vision direction is alternately turn 
left, forward and right. The motivation is given by boredom(t), if the prey is 
not moving and boredom(t)30.2, or zero, otherwise. The rest behavior makes the 
prey stop moving, with a motivation as for predators. 


Associative Learning 
The associative learning allows the prey to generalize spatial-temporal relations 


between external stimuli from particular instances. The mechanism is inspired on 
the neuroethological and semiotic constraints described previously, implementing 
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Fig. 2. (a) Associative learning architecture. (b) Association adjustment rules. 


a lower-order sensory domain through work memories and a higher order multi- 
modal domain by a associative memory (figure 2a). 

The work memories are temporary repositories of stimuli: when a sensorial 
stimulus is received from either sensor (auditory or visual), it is placed on the 
respective work memory with maximum strength, at every subsequent iteration 
it is lowered and when its strength gets to zero it is removed. The strength of 
stimuli in the work memory (WM) varies according to the expression 


1.0, if stimulus 7 arrived at sensors 
and it isn’t already 
in the work memory 
strengthW M;,(t) — 0.2, if stimulus 7 is in the work memory 


strengthW M,(t+1) = 


The items in the work memory are used by the associative memory to produce 
and update association between stimuli, following basic Hebbian learning (figure 
2b). When an item is received in the visual WM and in the auditory WM, an 
association is created or reinforced in the associative memory, and changes in 
its associative strength are inhibited. Inhibition avoids multiple adjustments in 
the same association caused by persisting items in the work memory. When 
an item is dropped from the work memory, its associations not inhibited, i.e. 
not already reinforced, are weakened, and the inhibited associations have their 
inhibition partially removed. When the two items of an inhibited association 
are removed, the association ends its inhibition, being subject again to changes 
in its strength. The reinforcement and weakening adjustments for non-inhibited 
associations, with strengths limited to the interval [0.0; 1.0], are done as follows: 

— reinforcement, given a visual stimulus 7 and a hearing stimulus 7 present in 
the work memories 


strengthi;(k + 1) =strengthi;(k) + 0.1(1.0—(maaxstrength,(k)—strengthi;(k))) 
+0.01 
where mazxstrength;(k) = maz; strengthi;(k) 
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— weakening, for every association related to the dropped visual stimuli i 


Vj associated with i, 
strengthi;(k + 1) = strength; (k) — 0.1(maxstrength;(k) — strengthi;(k))) 
+0.01 


— weakening, for every association related to the dropped hearing stimuli 7 


Vi associated with j, 
strengthi;(k + 1) = strength; (k) — 0.1(maxstrength;(k) — strengthi;(k))) 
+0.01 


As shown in figure 1, the associative learning can produce a feedback that in- 
directly affects drives and other behaviors. When an alarm is heard and it is 
associated with a predator, a new internal stimulus is created composed of the 
associated predator, the association strength, and the direction and distance of 
the alarm, which is used as an approximately location of the predator. This new 
stimulus will affect the fear drive and fleeing behavior. The fear drive is changed 
to account for this new information, which gradually changes fear value: 


ramp2(1.0), predator is seen 

ramp2(strengthi;(t)), predator is not seen, but an alarm 7 is heard 
and the strongest association is related to a predator j 

ramp2(fear(t) — 0.05(1.0 — fear(t)), otherwise 


fear(t+ 1) = 


This allows the associative learning to produce an escape response, even if 
the predator is not seen. This response is gradually learned and it describes a 
new action rule associating alarm with predator and subsequent fleeing behavior. 
The initial response to an alarm is a scanning behavior, typically indexical. If the 
alarm produces an escape response due to its mental association with a predator, 
our creature is using a symbol. 


3 Creatures in Operation 


The virtual environment inhabited by creatures works as a laboratory to study 
the conditions for symbol emergence. In order to evaluate our simulation ar- 
chitecture, we performed different experiments to observe the creatures during 
associative learning of stimuli. We simulate the communicative interactions be- 
tween preys in an environment with the different predators and objects, varying 
the quantity of creatures present in the environment. 

Initially, we used teacher and learner preys and change the number of teach- 
ers, predators and learners (figure 3). Results show that learners are always able 
to establish the correct associations between alarms and predators (alarm 1 - 
terrestrial predator, alarm 2 - aerial predator, alarm 3 - ground predator). The 
number of interactions decreased whereas the amount of competition among 
associations increased, as the number of teachers or predators increased. This 
is due to an increase in the numbers of vocalizing events from teachers, what 
corresponds to more events of reinforcement and less of weakening. Placing two 
learners in the environment, we could also notice that the trajectories described 
by the association values in each prey are quite different, partially because of 
random properties in their behavior. 
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(e) 


Fig. 3. Evolution of association strength values using Teachers and Learners (associ- 
ation value x iteration). Exp. A (1 learner (L), 5 teachers (T) and 3 predators (P)): 
associations with (a) alarm 1, (b) alarm 2 and (c) alarm 3. Exp. B (1 L, 5 T, 6 P): (d) 
winning associations for alarms. Exp. C (1 L, 10 T, 3 P): (e) winning assoc. for alarms. 
Exp. D (2 L, 5 T, 3 P): (d) winning associations in each creature. 


Using self-organizers, all preys can vocalize and learn alarms. Therefore, the 
number of alarms present in the simulations is not limited to three as before. 
Each prey can create a different alarm to the same predator and the one mostly 
used tends to dominate the preys’ repertoire at the end (figure 4). Increasing 
the number of preys, tends to increase the number of alarms, the number of 
interactions and also the amount of competition, since there are more preys 
creating alarms and also alarms have to disseminate among more preys. 

In a final experiment, we wanted to evaluate the adaptive advantage of using 
symbols instead of just indexes (figure 5). We adjusted our simulations by mod- 
elling an environment where visual cues are not always available, as predators, 
for instance, can hide themselves in the vegetation to approach preys unseen. 
This was done by including a probability of predators been actually seen even 
if they are in the sensory area. We then placed learner preys that responded to 
alarms by just performing scanning (indexical response) and preys that could 
respond to alarms using their learned associations (symbolic response). Results 
show that the symbolic response to alarm provides adaptive advantage, as the 
number of attacks suffered is consistently lower than otherwise. 


4 Conclusion 


Here we presented a methodology to simulate the emergence of symbols through 
communicative interactions among artificial creatures. We propose that symbols 
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Fig. 4. Evolution of association strength values for Self Organizers (mean value in 
the preys population). Exp. A (4 self-organizers (S) and 3 predators (P)): associations 
with (a) terrestrial predator, (b) aerial predator and (c) ground predator. Exp. B (8 S, 
3 Ppredators): associations with (d) terrestrial pred., (e) aerial pred. and (f) ground 
pred. 


Evaluation of fleeing response to alarms (attacks suffered) 


Fig. 5. Number of attacks suffered by preys responding indexically or symbolically to 
alarms. We simulated an environment where preys can’t easily see predators, introduc- 
ing a 25% probability of a predator being seen, even if it is within sensorial area. 


can result from the operation of simple associative learning mechanisms between 
external stimuli. Experiments show that learner preys are able to establish the 
correct associations between alarms and predators, after exposed to vocalization 
events. Self-organizers are also able to converge to a common repertoire, even 
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though there were no pre-defined alarm associations to be learned. Symbols 
learning and use also provide adaptive advantage to creatures when compared 
to indexical use of alarm calls. 

Although there have been other synthetic experiments simulating the devel- 
opment and evolution of sign systems, e.g. [4,6], this work is the first to deal 
with multiple distributed agents performing autonomous (self-controlled) com- 
municative interactions. Different from others, we don’t establish a pre-defined 
‘script’ of what happens in communicative acts, stating a sequence of fixed task 
to be performed by one speaker and one hearer. In our work, creatures can be 
speakers and/or hearers, vocalizing and hearing from many others at the same 
time, in various situations. 
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